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ABSTRACT 

Between 2006 and 2008 the Ministry of Education 
funded the evaluation of three Talent Development 
Initiatives for gifted and talented students. The 
methodology employed was one of participatory 
action research, a process of evaluation that 
enables learning by doing, as researchers and 
practitioners work alongside one another. Through 
the process of evaluating the three programmes, 
the researchers were also able to reflect on 
how and what formative feedback effects the 
development, implementation, maintenance, 
and evaluation of provisions for gifted and 
talented students. Based upon notes from team 
meetings, ongoing communications, and focus 
group discussions, the researchers concluded 
that an action research approach, despite some 
tensions, is useful and instrumental in the ongoing 
development of provisions for gifted and talented 
students. 
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INTRODUCTION 

The Talent Development Initiatives were launched 
by the previous government as one of a series 
of work programmes to enhance educational 
opportunities for gifted and talented students, 
their parents, and educators. Between 2003 and 
2008, two contestable funding rounds enabled 
over thirty-five providers to develop a range 
of initiatives. These included a wide diversity 
of innovative programmes, using different 
organisational structures, aimed at a variety of 
target groups and needs, provided by both schools 
and private providers. The original purposes in the 
contestable funding, as outlined by the (Ministry of 
Education, 2002), were to: 

• Develop innovative approaches to gifted 
and talented education that would result in 
improved outcomes for students. 

• Research the impact of innovative 
approaches on teaching and learning. 

• Disseminate knowledge, understanding, 
and models of effective practice. 


During the second round of funding, two types 
of initiatives were funded (Enhance and Ignite 
programmes) and a team of researchers was 
contracted to evaluate some of the initiatives. The 
Enhance programmes were selected for continued 
funding to further develop and build upon their 
innovations from the first round, and the Ignite 
programmes were new programmes with a targeted 
focus on specific groups of students (e.g., Maori 
and Pasifika, rural, low-income). Between 2006 
and 2008 the researchers evaluated two Enhance 
programmes of professional development and 
support for teachers using a case study approach, 
and three Ignite programmes designed for gifted 
and talented students using an action research 
approach. 

The evaluation of these five programmes has been 
reported by Riley and Moltzen (2010), and this 
article focuses on only one slice of the full report: 
action research to evaluate provisions for gifted 
and talented students. At the outset, it is important 
to clarify that we do not believe action research 
and evaluation research to be dichotomous or 
in conflict with each other. We also adopt the 
position of Potter (2006): there is no one correct 
way to conduct programme evaluation. Rather, 
we take the stance that the evaluation of gifted 
programmes should utilise a responsive research 
approach which incorporates collaboration, 
opportunities for improvement and change through 
an iterative process, and, ultimately, leads to 
empowerment of participants and sustainability of 
programmes. "Evaluation employs an inquiry cycle 
that iteratively frames and examines problems of 
practice, chooses actions to address the problems, 
assesses the effects of these actions, and then 
reframes the original problems of practice" (Rallis 
& Militello, 2009, p. 254). 

This definition of evaluation fits well with action 
research, which, as its name implies achieves 
"both action and research outcomes in a single 
study" (Dick, 1997, no page given). We believe 
that an action research evaluation encourages 
people to look at what is happening, reflect upon 
how programmes are developing, and make 
changes as they are going along, rather than 
just evaluating them at the end. In other words, 
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the focus of such an evaluation is on action or 
change, or continuous improvement. Potter (2006) 
would argue that the approach to evaluation 
we are describing is underpinned by a critical- 
emancipatory paradigm - one which aims for 
transformation through evaluation. 

The article begins by describing what we know 
about the evaluation of gifted programmes in 
New Zealand, followed by an explanation of 
the underlying principles and practices. The 
advantages and disadvantages of learning by 
doing, as perceived by the participants and 
researchers, are described, with recommendations 
for practice. 

WHAT DO WE KNOW? 

In New Zealand, gifted and talented provisions 
are seldom formally evaluated and reported 
upon, resulting in "a paucity of... research which 
evaluated the effectiveness of provisions ... in 
relation to social, cultural, emotional, creative, 
and intellectual outcomes" (Riley, et al., 2004, p. 
3). This lack of reported evaluations leaves New 
Zealand's practitioners in a vulnerable position 
of adapting or adopting international practices 
without a "critical, research-driven analysis of 
their appropriateness" (Riley & Moltzen, 2010, 
p. 5) within our own cultural and educational 
landscape. While there is a growing body of 
information about gifted and talented provisions, 
without empirical research evidence it is difficult 
for teachers to make informed decisions or reliably 
test the efficacy of chosen practices. 

And yet, what we know, from reports of the 
Education Review Office (2008) and Ministry of 
Education (Riley et al., 2004), is that only around a 
quarter of schools employ evaluation methods. In 
2007, the Education Review Office collected data 
on 315 schools, reporting that nearly half of the 
schools (46%) had not developed any processes of 
evaluation as would be indicated by: 

• systematic and ongoing processes for 
evaluating student outcomes; 

• sharing and consultation about evaluation 
findings with key stakeholders; 

• actions based on the recommendations of 
evaluations, and 

• evaluations of provisions both internal and 
external to the school. 

If these indicators were evidenced in schools or 
other services for gifted and talented students, 
chances are they would be better armed to 
make good decisions and to be responsive 
to students' needs. On the other hand, the 
lack of good evaluation data can result in an 


insufficient programme infrastructure which can 
potentially lead to superficial provisions (Van 
Tassel-Baska, 2004a). Therefore, in some ways 
it is not surprising, that after a three year action 
research-based evaluation of programmes for 
gifted and talented students, Riley and Moltzen 
(2010) concluded that each of the programmes 
demonstrated a dynamic approach to programme 
development, implementation, and evaluation. 
Using formative evaluations, the providers were 
able to collect information from key stakeholders 
and respond to their findings to improve their 
provisions. This needs-based approach requires 
"flexibility, adaptability, creativity, and innovation" 
(Riley & Moltzen, 2010, p. 143). These same 
factors are important in evaluating provisions for 
gifted and talented students, as the next section 
explains. 

WHAT DO WE NEED TO KNOW? 

There are two interconnected elements of 
evaluation: the outcomes for students and the 
effectiveness of the programme (Taylor, 2000). 
Academic, social, cultural, emotional, creative, 
and intellectual outcomes contribute to a 
programme's overall effectiveness, and, similarly, 
the effectiveness is in one part determined by 
evidenced outcomes for students. The effective 
implementation of a programme is determined 
by measuring the effectiveness of management 
objectives, while the programme objectives 
determine outcomes for students (Gallagher, 1998). 
Thus, provisions need to be measured based on 
input (i.e., resources), processes (i.e., identification 
and differentiated programmes), and outputs 
(i.e., student outcomes) (Taylor, 2000). To do this, 
evaluation needs to be ongoing and based on 
formative and summative approaches. 

The National Association for Gifted Children in the 
United States (1997) suggests different approaches 
to evaluation: incoming evaluations, or needs 
analyses; transition evaluations (when students 
move from one programme to another); year-end 
evaluations; and ongoing evaluations. Ongoing 
evaluations enable timeliness: a chance to measure 
the process of development, to test innovation, and 
gather formative data (Van Tassel-Baska, 2004b). 
This requires an eclectic mix of evaluation types, 
as described by Van Tassel-Baska (2004b): 

• Case study approaches answer the questions 
of how and why by providing rich, thick 
descriptive data from multiple perspectives. 

• Utilisation-focused approaches embed 
evaluation methods into programme 
development and implementation, enabling 
the data to influence programme changes. 
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• Client-centred approaches focus on the 
needs of a provision's key stakeholders 
by enabling their direct involvement in 
shaping, contributing to, and providing 
feedback for the evaluation. 

• Context, input, process, product 
approaches measure the extent to which 
the programme's goals have been met by 
considering how these four factors have 
been evidenced in the programme planning, 
documentation, and procedures. 

Ideally, any evaluation of provisions for gifted and 
talented students would take on a mix of these 
approaches, weaving together all these foci by 
gathering data from multiple sources and analysing 
these both quantitatively and qualitatively. Using 
interviews, observations, document analysis, 
surveys, and outcome-based evidence enables 
evaluators to triangulate different data sources 
from different perspectives, as well as substantiate 
findings through recurring themes, patterns, or 
issues. 

To do this requires a team approach, inclusive 
of the many stakeholders in gifted and talented 
programmes: administrators, teachers, parents 
and whanau, community members, and the 
students themselves (Riley & Moltzen, 2010). 
Tomlinson and Callahan (1994) also recommend 
the inclusion of qualified evaluators - experienced, 
skilled, and knowledgeable 'experts'. Each of 
these stakeholders will have different evaluative 
questions, interests, and purposes (Reid, 

2004), but by including many perspectives, 
the evaluation will generate greater support, 
shared understandings, and opportunities for 
implementing recommendations (Tomlinson & 
Callahan, 1994). As Van Tassel-Baska (2004b) 
states, "Involvement increases relevance, 
understanding, and ownership of the evaluation, 
all of which facilitate informed and appropriate 
use" (p. 20) of the results. 

EHowever, evaluation is complex and not without 
problems. Difficulties in the evaluation of gifted 
and talented programmes stem from deeper issues 
in the field. For example, differences in definitions 
and identification of giftedness and talent 
means that any generalisations across different 
evaluation approaches or results should only 
be made with caution. Measuring multi-faceted 
student outcomes, including emotional, social, 
and cultural outcomes, is unachievable using 
standard measures of assessment: alternative and 
above-level measures need to be utilised. Given 
the nature of some gifted programmes and the 
individuality of students, it would be difficult to 
pinpoint 'the programme' as the sole cause for any 
change or growth. As Winner (1996) explained, 


this could only be achieved by random sampling 
- identifying students as gifted and talented and 
then implementing an intervention with one group 
(treatment) but not another (control). As Reid 
(2004) reminds us, research of this nature creates 
ethical dilemmas. Therefore, in the evaluation of 
gifted and talented programmes, it is important 
to take individual differences into consideration, 
avoid sweeping generalisations or cause-effect 
conclusions, and employ appropriate measures. 

To summarise, evaluations of gifted and talented 
programmes should provide information for 
improvements using a collaborative process to 
gather data from multiple sources (Van Tassel- 
Baska, 2004c). What this enables is a dynamic 
interplay between programme planning and 
development, implementation, and evaluation 
which Van Tassel-Baska (2004c) describes as a 
cyclical process of planning, doing, studying, 
and acting. Taking all these considerations into 
account, it makes sense to use an action research 
approach. 

LEARNING BY DOING 

Participatory action research is "a systematic 
inquiry by collaborative, self-critical communities 
... out of the need to improve educational 
knowledge and practices" (Watts & Watts, 1993, 
p. 36). Grundy (1982) described the underlying 
principles of participatory action research as 
collaboration and participation, empowerment, 
knowledge, and social change. Through authentic 
and committed participation, communities of 
people open themselves up to enlightenment 
about the relationships between "circumstance, 
action, and consequence" (Wadsworth, 1997, p. 
70). Through dissemination and sharing of their 
experiences, this enlightenment is extended to 
other stakeholder groups. These ideas align well 
with what we know about the evaluation of gifted 
and talented programmes, and particularly the 
testing of innovative approaches. 

What an action research approach entails is a 
spiral of steps taken by teams of researchers and 
programme stakeholders. These steps can be 
applied to the evaluation of provisions as follows: 

1. Develop a plan of action for evaluating 
outcomes and processes. 

2. Act to implement the plan by undertaking 
the evaluation. 

3. Observe the effects of the action by 
documenting the evaluation. 

4. Reflect on the action and plan further 
action for the evaluation. 
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By using this sort of approach, an evaluation would 
be both responsive and illuminative. A major 
benefit is that the evaluation is not just summative, 
but also formative, enabling "judgement of the 
worth of the programme while it is forming or 
happening. It also allows for the programme to be 
shaped based on an ongoing evaluation, rather 
than a 'too-late' evaluation" (Riley & Moltzen, 

2010, p. 12). Using this design also allows 
providers to build records of improvement and, if 
planned at the outset of programme development, 
to tell the story of programme evolution. 

For the evaluation of the Talent Development 
Initiatives for gifted and talented students, an 
action research approach was employed over 
three years with teams of researchers working 
alongside practitioners. Each team included three 
academics with experience and knowledge in 
gifted and talented education, as well as curricular 
knowledge related to subjects or levels of teaching. 
The initiative's director(s) were also part of each 
team. 

The teams worked together to develop processes 
for determining the most appropriate approaches 
and perspectives for answering the Ministry of 
Education's research purposes: 

• To determine how providers design, 
implement, maintain, and evaluate 
programmes for gifted and talented students. 

• To determine how providers structure 
relevant and engaging learning and growth 
opportunities for gifted and talented 
students, as evidenced in the achievement 
of programme objectives; improved 
outcomes for students; impact upon key 
stakeholders, and planning for sustainability. 

• To determine how, and what, formative 
feedback effects the development, 
implementation, maintenance, and 
evaluation of programmes for gifted and 
talented students, by using an action 
research approach to evaluation (Riley & 
Moltzen, 2010, p. 2). 

Thus, each year represented a cycle of research 
framed by a broad question: 

1. Cycle 1: What is going on? 

2. Cycle 2: Is it working? 

3. Cycle 3: EHow do we know? 

In other words, during the first year of programme 
development, the focus was on programme design 
and implementation. As researchers, who were 
external to the conceptualisation and creation 
of the programmes, it was important to gain a 
sense of how students were being identified: 
what curriculum adaptations were being made; 


how the content, processes, and products were 
differentiated, and how the providers would 
determine programme effectiveness with their 
own internal processes. By the second cycle, it 
was time to focus upon how those decisions were 
working in practice: were they effective measures 
of identification, differentiated provisions, and 
so on? In the final year, while effectiveness was 
still of great importance, it was also timely to 
ask questions related to sustainability, evaluation 
processes in practice, personnel, and programme 
impacts. 

Each of the three case studies was evaluated using 
several methods of data collection: document 
analysis; observation; surveys, and interviews 
(focus group and individual). Perspectives were 
sought from all stakeholders, including gifted 
and talented students, the programme director(s), 
teachers, mentors, and parents. For each case study 
different data methods were used with different 
stakeholder groups, but all participants were well- 
informed of the reasons for and implementation of 
action research in the evaluation. All participants 
were also provided with a detailed information 
sheet and asked to provide informed consent, 
as per the Massey University EHuman Ethics 
Committee guidelines. 

Throughout the three years of the evaluation, 
the research teams met regularly, communicated 
over email and phone, and kept research diaries. 
During the final research team meeting in 2009, a 
doctoral student was hired to conduct focus group 
interviews with the programme directors and the 
academic researchers, probing their experiences 
and perspectives as members of an action 
research-driven evaluation of gifted and talented 
programmes. From these sources, the researchers 
were able to contemplate the usefulness of action 
research for the evaluation of gifted and talented 
programmes. 

WHAT DID WE LEARN? 

As Van Tassel-Baska (2004c) promised, by 
employing an action-oriented evaluation method, 
the researchers and programme directors were 
able to plan, do, study, and act as these innovative 
initiatives evolved. Working alongside one another 
through the research cycles enabled planning, 
implementation, evaluation, and then the creation 
of a plan of action for improvement. Over the three 
years, this approach ensured collaboration and 
participation, empowerment, and change (Grundy, 
1982). For new programmes, which were evolving 
and changing as they were being implemented, 
this approach proved useful; however, it was not 
always smooth sailing. 
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Also, while it should be noted that some of the 
perceived advantages and disadvantages of this 
approach may relate to other ways of conducting 
evaluations, what was unique in this project was 
the purposeful mixing of 'expert' researchers and 
'expert' practitioners (Riley & Moltzen, 2010). 
Using an action-driven method which emphasised 
change led to fluidity and responsiveness in the 
evaluations. For example, although the researchers 
were contracted by the Ministry of Education to 
ask a specified set of research questions, there was 
scope to be responsive and flexible in the ways 
in which those were pursued. As the programmes 
were shaped and evolved, so, too, did the 
evaluation. 

Advantages of Learning by Doing 

Riley and Moltzen (2010) reported an array 
of advantages for stakeholders in each of the 
programmes and including the researchers. For 
example, the use of a collaborative approach gave 
key stakeholders, including students and parents, 
opportunities to have a 'voice' and to influence 
programme development and implementation. 

For the gifted and talented students, "it provided 
opportunities for sharing their work and ideas in 
a broader context" (Riley & Moltzen, 2010, p. 

145). Using multiple measures of effectiveness 
from different perspectives ensured validation 
and triangulation of emerging and final results. 
Some of the advantages for the programme 
directors involved in this project included: access 
to relevant expert knowledge, materials, and 
resources; support, encouragement, and validation 
of practices, including wider dissemination, and 
professional learning and growth. Each of these 
advantages is elaborated in the following sections. 

The programme directors had access to relevant 
expert knowledge by careful matching of research 
team members with programmes. For example, 
a visual arts programme was evaluated by an 
academic in gifted and talented education, 
supported by a music educator and visual artist, 
both of whom had professional interests in gifted 
and talented students. While the lead researcher 
was able to provide expertise on 'the big picture', 
his supporting artists were able to add a touch 
of fine detail, depth, and experience specifically 
related to the programme. Similarly, a marine 
biology programme was supported by a generalist 
in gifted and talented education, a science teacher 
educator, and a mathematics teacher educator. 

Materials, resources, and expertise offered by 
universities also provided an advantage for 
the programme directors. As members of the 
evaluation team, the programme directors had 
access to library resources, including books 


and journals, which they otherwise would not 
find in their schools or centres. In one case, the 
programme director chose to spend several days 
at the Massey University campus so that he could 
refine his evaluative tools. This gave him access to 
expertise in terms of input and resources, but also 
the 'space' in which to focus on his work, free of 
distractions normally associated with his role in the 
centre. 

Working with the researchers allowed 
opportunities for support, encouragement, 
insights, and validation of practice. Planning 
and implementing specialist programmes can 
sometimes be a rather isolated experience; 
however, being part of a research team provided 
collegial and professional support. For example, 
the director of a school-based programme 
received limited 'in-house' support, as no formal 
mechanisms were in place to offer such. In this 
case, the academic researchers "probably assumed 
a more influential role than might be expected 
to normally occur" (Riley & Moltzen, 2010, p. 

76). The director explained "They'd basically 
walk in the door by the end and it was just verbal 
diarrhoea from me most of the time - they were 
just the only people I could talk to, 'cos I'm quite 
isolated with what I do, it was like 'blah blah blah 
blah'" (Focus Group Interview, 2009). Another 
director said, "They fitted in beautifully, they fed 
back data, they've been very sharing. I couldn't 
have asked for a better team and they've been very 
much mentors" (Focus Group Interview, 2009). 

Assistance and support in ongoing dissemination 
of practice was a real benefit for the directors. 

The programme directors presented nationally 
and internationally at conferences, sometimes 
in collaboration with the academic team 
members. Collaborative writing projects have 
also eventuated. As one of the directors, who had 
presented a workshop with one of the researchers, 
shared, "I think that was a huge boost for me, a 
learning curve for me" (Focus Group Interview, 
2009). Riley et al., (2004) concluded that the 
dissemination of New Zealand-based research in 
gifted and talented education is found lacking, 
and furthermore, when it is disseminated, it 
doesn't always reach practitioners. Each of these 
programmes has reached a range of local, national, 
and international audiences through presentations 
and publications. This has also served to enhance 
their own processional learning and support (Riley 
& Moltzen, 2010). 

Flaving an external perspective to programme 
development, implementation, and evaluation 
was also seen as valuable. As was stated 
earlier, programme coordinators can sometimes 
experience feelings of isolation, and as a 
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programme develops, this can feel a bit like navel 
gazing! The academic researchers did not have 
the same sense of ownership of the programmes, 
and this afforded opportunities to provide 
different practical, theoretical, and research- 
based ideas. The researchers could also ask the 
'tough questions', enabling critical analysis of the 
programmes as they evolved. In the focus group 
discussion (2009), one of the directors explained it 
like this: 

And it's been another set of eyes ... 
because once you get really close to 
something you can often be a little bit 
blind and blinkered to things and it's 
been really important to have someone 
else see it from another perspective. 

As a result of all these advantages, each director 
also had opportunities for, and showed evidence 
of, professional growth. As one of the directors 
shared in the focus group interview (2009), 

As part of the process of doing ... 
your own professional development is 
happening - whether you want it to or 
not, in a sense. And so it's like whoever 
you're working with, you're moving 
them along, and ... you are having to 
move along yourself. 

This seems unique to the methodology, as the 
hands-on involvement in the evaluation ensured 
the directors were not 'outside' the process, results, 
or conclusions, but firmly cemented within the 
action research. 

Similarly, there were advantages for the academic 
researchers. Firstly, there was the opportunity to 
be engaged in practice which created "practice- 
informed research, an often overlooked link 
between theory and practice whereby research 
usually drives practice" (Riley & Moltzen, 2010, 
p. 145). Being external to the programme's 
conceptualisation, and to a lesser degree its 
implementation, meant that any research or 
theory-driven ideas really had to be contemplated 
in relation to practice, and considerate of 
programme development. As one of the research 
team members explained, "It made me think, if 
I'm evaluating, at the same time 'How can I feed¬ 
forward for the people I'm evaluating, working 
alongside them, that helps them move forward 
and then spurs another bit of growth?"' (Focus 
Group Interview, 2009). This responsiveness took 
at least one researcher a while to grow accustom 
to: she referred to this as "delayed resolution as a 
researcher" (Focus Group Interview, 2009). 

Also, involvement in practice-driven research 
influenced tertiary teaching by developing 
understandings of giftedness, provisions, and 


evaluation, and providing examples of practice. 
One researcher explained it like this: 

It was kind of a reinforcement of 
characteristics if you like, of gifted and 
talented students, the importance of 
working with like-minded peers, and I 
think the key for me, the biggest growth, 
was my understanding of how mentors 
could be used (Focus Group Interview, 
2009). 

Another researcher felt her involvement in a 
secondary programme added to her "credibility" as 
a lecturer with a background in primary education. 
Research skills and abilities were also developed 
and there was great value placed on working as 
part of a team with a balance of experiences, skills, 
and backgrounds. 

There were clearly positive spin-offs for all those 
involved in the evaluations, and the researchers 
became a natural part of the Talent Development 
Initiatives. But this was not without tensions and 
issues, especially for the academic researchers, 
which the next section describes. 

Tensions for Researchers 

From the outset of these evaluations, it was 
important that there be an established level of 
trust between the researchers and the programme 
stakeholders. In initial discussions with the 
programme providers, while on the one hand they 
were willing to cooperate, on the other hand it 
was not without some scepticism, wariness, and 
possibly even fear of criticism. Action research 
requires the development of collaborative, 
trusting and respectful relationships, but this 
was heightened by the fact that both the Talent 
Development Initiatives and the evaluation 
approach were innovative and still under 
development. So, in a sense, not only were the 
programmes evolving, so too was the research. 

Not surprisingly then, finding a balance between 
the research process and final product was at 
times difficult for the researchers. How important 
was it to help develop, shape, and even influence 
the programme while at the same time pursuing 
the research questions? Similarly, it was at times 
difficult to maintain and demonstrate respect and 
understanding for the programme's circumstances 
and needs, while operating within a defined set 
of research parameters. Riley and Moltzen (2010) 
describe this as a tension between 'bottom-up' or 
'top-down' approaches to research. For example, 
one of the research team members described her 
dilemma when asked to conduct parent meetings, 
and being advised by one of the research directors 
not to do so: "I would have fallen into the trap 
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of being more involved in the process of the 
programme rather than knowing quite where to 
draw the line" (Focus Group Interview, 2009). 

It was also difficult defining the roles of the 
researchers and the participants. These roles shifted 
throughout the research, ranging from novice 
to expert, researcher to practitioner, and insider 
to outsider. The researchers were academics 
with expertise in theory and research, while the 
programme directors were expert practitioners. 

But even within the research teams, there was a 
complexity of expertise in both research methods 
and contexts. Over the three years of the research, 
however, all these roles shifted. The programme 
directors were defined initially by their teaching 
practice, but were expected to become active 
researchers. Similarly, the researchers were marked 
by their academic expertise and expected to 
become involved in practice. 

As the action research progressed, the lines 
between researchers and practitioners became 
fuzzy and expertise more balanced. For example, 
as questions and problems arose, solution-finding 
became more collaborative, particularly as each 
cycle of the research progressed. A researcher 
described the changing roles in this way: "And 
it's also that they feed back and you work 
collaboratively with them and then something new 
evolves from it and the pathway is explicit and 
obvious" (Focus Group Interview, 2009). There 
was greater sharing of the ownership of both the 
research and programmes' problems. But this also 
caused dilemmas: "Yeah there were problems 
with boundaries, with boundaries around the 
programme" (Researcher, Focus Group Interview, 
2009). 

As has been described, both the researchers and 
programme directors demonstrated professional 
growth, and this aided in the fluidity of roles. 
Across and within the teams the research roles and 
programme roles became more transferable and 
interchangeable. Building on individuals' strengths 
enabled a collaborative sharing of tasks as both 
problem-finders and problem-solvers. While this 
sounds positive, and to a large degree it was, for 
the researchers it created a tension between the 
roles of collaborator and external evaluator. The 
researchers were acting in two roles, insiders and 
outsiders, and there was some danger in objectivity 
being lost as relationships developed. As one of the 
research team members explained: 

I found a real tension for me between 
being an evaluator and what I consider 
my role as an evaluator of the project, 
relative to what the project goals were, 
and, so I was really on the outside 
looking in, but I couldn't help but get 


Weaving educational threads. Weaving educational practice. 


on the inside and be a mentor so there 
was a sort of real tension between 
"Well, how much am I supposed to 
be taking on a role of being part of 
the team?" to feed-back, feed-forward 
(Focus Group Interview, 2009). 

It was also difficult at times to be critical of the 
programmes, especially when sometimes these 
were derived from theory and research developed 
by members of the research teams. It was also 
difficult to know when to be hands-on or hands- 
off. Another team member described the conflict 
like this: "that grey area ... where we're outside 
the organic model so to speak versus how much 
we put inside and lead" (Focus Group Interview, 
2009). There were role conflicts that evolved: 
researcher versus expert, researcher versus monitor, 
and supporter versus advocate. 

WHERE TO FROM HERE? 

Programmes for gifted and talented students 
should be evaluated both formatively and 
summatively, using internal and external processes, 
and inclusive of a variety of stakeholder voices 
heard through multiple measures. One important 
element that is sometimes overlooked is the 
importance of establishing the framework and 
design of an evaluation at the outset of programme 
development. Using an action research approach 
provides the flexibility and fluidity needed for 
parallel evolution of the programme and the 
evaluation. 

More research is needed to test the viability 
and strength of action research approaches to 
programme evaluation in gifted and talented 
education, but from this preliminary work, a set of 
practical recommendations for evaluation arise. 
When designing and implementing programme 
evaluations for gifted and talented education, 
educators should consider: 

1. Using a team approach which is inclusive 
of stakeholders from within the school 
community, but also professionals external 
to the school who may have expertise. 

2. Carefully matching the evaluation purposes 
with the methods of data collection. 

3. Developing trust and clarifying roles of all 
those involved in the evaluation process. 

4. Using the results of evaluation for 
programme improvement; this requires 
gathering practical information by asking 
practice-driven questions. 
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